Skip to content

Conversation

@james-willis
Copy link
Contributor

This PR sets up zarr-layer so that it correctly gets chunks from pyramids when those pyramid layers don't span the whole globe. I'm working on a system that generates multiscale pyramids like ndpyramid, but it only defines the chunks for tiles which overlap the AOI of the original zarr at each zoom. so while z=0 will always cover the whole globe, some dataset in St Paul will only over (1, 0, 0) at z=1.

This PR is a bit long because it handles several cases:

  1. With or without consolidated metadata: without consolidated metadata we use a lazy approach because otherwise it creates too much start up lag.
  2. with or without spatial_ref: I would think spatial ref would always be present but we have a fallback case when its not. This is much slower because there are so many network requests to the coordinate arrays.
  3. zarr 2 vs 3

@vercel
Copy link

vercel bot commented Jan 1, 2026

@james-willis is attempting to deploy a commit to the carbonplan Team on Vercel.

A member of the Team first needs to authorize it.

@james-willis james-willis marked this pull request as ready for review January 1, 2026 02:59
@Shane98c
Copy link
Member

Shane98c commented Jan 5, 2026

thanks for this! Would it be possible to provide an example dataset to test alongside this? I'm curious what benefits a pyramid of this shape brings that a store with sparse chunk storage that just 404s on chunks outside of the extent doesn't. I'm hesitant to diverge away from our standard slippy map xyz key => chunk indices for the tile-mode path.

@vercel
Copy link

vercel bot commented Jan 5, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Review Updated (UTC)
zarr-layer Ready Ready Preview, Comment Jan 5, 2026 9:23pm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants